Topic distillation via subsite retrieval

ABSTRACT

A method and system for generating a search result for a query of hierarchically organized documents based on retrieval of subtrees that are key resources for topic distillation is provided. The retrieval system may identify documents relevant to a query using conventional searching techniques. The retrieval system then calculates a subtree feature for subtrees that have an identified document as their root. After the retrieval system calculates the subtree feature for the subtrees, the retrieval system may generate a subtree relevance score for each subtree based on its subtree feature. The retrieval system may then order the identified documents based on their corresponding subtree relevances.

BACKGROUND

Many search engine services, such as Google and Overture, provide forsearching for information that is accessible via the Internet. Thesesearch engine services allow users to search for display pages, such asweb pages, that may be of interest to users. After a user submits asearch request (i.e., a query) that includes search terms, the searchengine service identifies web pages that may be related to those searchterms. To quickly identify related web pages, the search engine servicesmay maintain a mapping of keywords to web pages. This mapping may begenerated by “crawling” the web (i.e., the World Wide Web) to identifythe keywords of each web page. To crawl the web, a search engine servicemay use a list of root web pages to identify all web pages that areaccessible through those root web pages. The keywords of any particularweb page can be identified using various well-known informationretrieval techniques, such as identifying the words of a headline, thewords supplied in the metadata of the web page, the words that arehighlighted, and so on. The search engine service may generate arelevance score to indicate how relevant the information of the web pagemay be to the search request based on various metrics such as the termfrequency by inverse document frequency metric (“tf*idf”). The searchengine service may also generate an importance score to indicate theimportance of the web page based on various metrics such as Google'sPageRank metric. The search engine service then displays to the userlinks to those web pages in an order that is based on a rankingdetermined by their relevance and importance.

Two well-known techniques for determining the importance of web pagesare PageRank and HITS (“Hyperlink-Induced Topic Search”). PageRank isbased on the principle that web pages will have links to (i.e.,“outgoing links”) important web pages. Thus, the importance of a webpage is based on the number and importance of other web pages that linkto that web page (i.e., “incoming links”). In a simple form, the linksbetween web pages can be represented by matrix A, where A_(ij)represents the number of outgoing links from web page i to web page j.The importance score w_(j) for web page j can be represented by thefollowing equation:w _(j)=Σ_(i) A _(ij) w _(i)

This equation can be solved by iterative calculations based on thefollowing equation:A ^(T) w=wwhere w is the vector of importance scores for the web pages and is theprincipal eigenvector of A^(T).

The HITS technique is additionally based on the principle that a webpage that has many links to other important web pages may itself beimportant. Thus, HITS divides “importance” of web pages into two relatedattributes: “hub” and “authority.” “Hub” is measured by the “authority”score of the web pages that a web page links to, and “authority” ismeasured by the “hub” score of the web pages that link to the web page.In contrast to PageRank, which calculates the importance of web pagesindependently from the query, HITS calculates importance based on theweb pages of the result and web pages that are related to the web pagesof the result by following incoming and outgoing links. HITS submits aquery to a search engine service and uses the web pages of the result asthe initial set of web pages. HITS adds to the set those web pages thatare the destinations of incoming links and those web pages that are thesources of outgoing links of the web pages of the result. HITS thencalculates the authority and hub score of each web page using aniterative algorithm. The authority and hub scores can be represented bythe following equations:${a(p)} = {{\sum\limits_{q->p}{{h(q)}\quad{and}\quad{h(p)}}} = {\sum\limits_{p->q}{a(q)}}}$where a(p) represents the authority score for web page p and h(p)represents the hub score for web page p. HITS uses an adjacency matrix Ato represent the links. The adjacency matrix is represented by thefollowing equation: $b_{ij} = \left\{ \begin{matrix}1 & {{if}\quad{page}\quad i\quad{has}\quad a\quad{link}\quad{to}\quad{page}\quad j} \\0 & {otherwise}\end{matrix} \right.$

The vectors a and h correspond to the authority and hub scores,respectively, of all web pages in the set and can be represented by thefollowing equations:a=A ^(T) h and h=Aa

Thus, a and h are eigenvectors of matrices A^(T)A and AA^(T). HITS mayalso be modified to factor in the popularity of a web page as measuredby the number of visits. Based on an analysis of click-through data,b_(ij) of the adjacency matrix can be increased whenever a user travelsfrom web page i to web page j.

Although these techniques are useful for locating a broad range ofdocuments that are relevant to a query, these techniques often do notprovide satisfactory results for a “topic distillation.” Topicdistillation refers to the finding of key resources for a particulartopic. Some surveys have indicated that in 30%-40% of web searches theuser is interested in finding key web pages for the topic of the query.Although a search engine may return a broad range of documents thatinclude key documents, those key documents may be buried deep within thethousands of documents of the search result. For example, a person whosubmits the query “wireless communication” may want to identify a keyweb page for this topic. A typical search engine that bases relevance onthe content of each web page may include the web pages of Table 1 in thesearch result. TABLE 1 Rank Document ID Relevance URL 70 G35-97-10565619.858 cio.doe.gov/wireless/3g/3g_index.htm 470 G07-38-3990160 9.508cio.doe.gov/spectrum/groups.htm 477 G35-75-1119753 9.481cio.doe.gov/spectrum/philo.htm 518 G36-35-1278614 9.320cio.doe.gov/wireless/background.htm 571 G07-10-2999356 9.093cio.doe.gov/spectrum/background.htm 648 G35-01-1537522 8.817cio.doe.gov/wireless/wwg/wwg_index.htm 649 G07-78-3824915 8.815cio.doe.gov/wireless/

Table 1 lists web pages of the search result that are within thesubsites “cio.doe.gov/wireless” and “cio.doe.gov/spectrum.” Since theuser is interested in key web pages relating to wireless communication,the web page identified with the URL “cio.doe.gov/wireless” is likelymore of a key resource to the topic of “wireless communication” than theother web pages listed in Table 1. However, because the relevance to thequery may be based on the similarity of the content of the web pages tothe query, many web pages that are not key resources for topicdistillation have a higher relevance score and thus a higher ranking. Insuch a case, it can be difficult for a user to search through the pagesof search results to identify key resources to the topic of interest.

SUMMARY

A method and system for generating a search result for a query ofhierarchically organized documents based on retrieval of subtrees thatare key resources for topic distillation is provided. The retrievalsystem may identify documents relevant to a query using conventionalsearching techniques. The retrieval system then calculates a subtreefeature for subtrees that have an identified document as their root. Thesubtree feature is a combination of a contribution of that featurederived from the root document along with a contribution of that featurederived from the descendant documents of that root document. After theretrieval system calculates the subtree feature for the subtrees, theretrieval system may generate a subtree relevance score for each subtreebased on its subtree feature. The retrieval system may then order theidentified documents based on their corresponding subtree relevances.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram that illustrates the hierarchical relations of webpages of a sample web site.

FIG. 2 is a block diagram that illustrates components of the retrievalsystem in one embodiment.

FIG. 3 is a flow diagram that illustrates the processing of the re-rankresult using subsite component of the retrieval system in oneembodiment.

FIG. 4 is a flow diagram that illustrates the processing of thecalculate subsite feature component of the retrieval system in oneembodiment.

FIG. 5 is a flow diagram that illustrates the processing of thecalculate descendant contributions component of the retrieval system inone embodiment.

FIG. 6 is a flow diagram that illustrates the processing of thecalculate descendant contributions at level component of the retrievalsystem in one embodiment.

FIG. 7 is a flow diagram that illustrates the processing of the rankancestor subsites component of the retrieval system in one embodiment.

DETAILED DESCRIPTION

A method and system for generating a search result for a query ofhierarchically organized documents based on retrieval of subtrees thatare key resources for topic distillation is provided. In one embodiment,the retrieval system identifies documents relevant to a query usingconventional searching techniques. For example, the retrieval system mayinput a query, submit the query to a search engine service, and receivethe search result from the search engine service. The retrieval systemthen calculates a subtree feature for subtrees that have an identifieddocument as its root. The subtree feature is a combination of acontribution of that feature derived from the root document along with acontribution of that feature derived from the descendant documents ofthat root document. For example, the feature may be a term frequency byinverse document frequency metric or any other feature or combination offeatures that can be used to determine relevance of a document to aquery. After the retrieval system calculates the subtree feature for thesubtrees, the retrieval system may generate a subtree relevance scorefor each subtree based on its subtree feature. Since the subtreerelevance score is generated from features aggregated from both the rootdocument and its descendant documents, the subtree relevance scoreprovides a more accurate representation of the relevance of the rootdocument to the query for purposes of topic distillation. The retrievalsystem may then order the identified documents based on theircorresponding subtree relevances. The retrieval system may also assessthe subtree relevance of ancestor documents of the identified documents.The retrieval system may calculate the subtree relevance for the parentdocument, the grandparent document, and so forth. The retrieval systemmay then select the ancestor document (one may consider a document to beits own ancestor) with the highest subtree relevance for inclusion inthe search result. In this way, the retrieval system provides searchresults that identify subtrees of documents that represent key resourcesof the query topic, rather than results that identify individualdocuments whose content is more relevant to a query than the content ofother documents.

In one embodiment, the retrieval system interfaces with a web-basedsearch engine to identify subsites (i.e., a portion of a web site) ofweb pages that are key sources of information for a query topic. Theretrieval system considers each web page of the search result to be aroot web page of a subsite. The retrieval system calculates a subsitefeature by combining a contribution of the feature of the root web pageand a contribution of the feature derived from the descendant web pages.When calculating the contribution of the descendant web pages, theretrieval system weights the contribution of descendants with a closeancestral distance more highly than the contribution of descendants witha distant ancestral distance. For example, the contribution of a featurederived from child web pages of a root web page will have a greatereffect on the subsite feature than the contribution derived fromgreat-grandchildren web pages. In addition, the retrieval system weightsthe contribution of descendants that have fewer siblings more highlythen the contribution of descendants with more siblings. For example,the contribution of a feature derived from a child web page with onlyone sibling will have a greater effect on the subsite feature than thecontribution derived from a child web page with 10 siblings. In thisway, the retrieval system factors in both ancestral distance and numberof siblings in determining contribution of a web page to a subsitefeature.

In one embodiment, the retrieval system uses an ancestral distancepunishment factor when determining the contribution of a web page. Theancestral distance punishment factor may help prevent “concept drift”that results from more and more levels of web pages being added to asubsite. The retrieval system may represent the ancestral distancepunishment factor as follows:α(Δl)=λ^(Δl)  (1)where a represents an ancestral distance punishment factor, Δlrepresents an ancestral distance between a root web page and adescendant web page, and λ represents a control parameter to control theamount of punishment. The ancestral distance is the number of levelsfrom a web page to a descendant web page. The level l of a web page of aweb site corresponds to its depth within the subsite. The root web pageof a subsite has a level of 1, its children web pages have a level of 2,their children's web pages have a level of 3, and so on. Thus, theancestral distance between web pages is represented as follows:Δl=l(a)−l(b)  (2)where l(x) represents the level of web page x, and a and b represent webpage a and its descendant web page b. By adjusting the controlparameter, the retrieval system can affect the contribution of ancestorweb pages to a subsite feature. For example, if the control parameter isset to ½, then the punishment factor for child web pages will be ½, forgrandchild web pages will be ¼, for great-grandchild web pages will be⅛, and so on. By increasing the control parameter, the effect of moreancestrally distant web pages will increase, and vice versa.

In one embodiment, the retrieval system uses a sibling punishment factorwhen determining the contribution of a descendant web page. The siblingpunishment factor may help prevent the contribution resulting from alarge number of sibling web pages at a certain level from overwhelmingthe contribution of the root web page of the subsite. If a web page has10 child web pages, then even though the contribution from each childweb page is punished by the ancestral distance punishment factor, theoverall contribution from so many child web pages may be large. Theretrieval system decreases the contribution of a web page as its numberof sibling web pages increases. The retrieval system may represent thesibling punishment factor as follows: $\begin{matrix}{\frac{1}{{R\left( p_{s} \right)}}{\sum\limits_{u = 1}^{{h{(p_{s})}} - 1}\left\lbrack {\sum\limits_{p_{i_{1}} \in {R{(p_{s})}}}{\sum\limits_{p_{i_{2}} \in {R{(p_{i_{1}})}}}{\cdots{\sum\limits_{p_{i_{u}} \in {R{(p_{i_{u - 1}})}}}\frac{f\left( p_{i_{u}} \right)}{\prod\limits_{k = 1}^{u}{{R\left( p_{i_{k}} \right)}}}}}}} \right\rbrack}} & (3)\end{matrix}$where h(p_(s)) represents height of the subsite with p_(s) as the rootweb page, R(p_(s)) represents the child web pages of p_(s), ∥a∥represents the number of elements of a, and f(p_(i) _(u) ) representsthe feature of web page p_(i) _(u) . As indicated by Equation 3, theretrieval system sums the contribution of each descendant web pagepunished by the product of the number of siblings at each higher levelin the subsite. For example, the contribution of a web page at level 3with 5 siblings with a parent web page at level 2 with 4 siblings is1/20 (i.e., 1/(4*5)). The retrieval system may further punish theresulting contribution by the number of child web pages.

In one embodiment, the retrieval system may combine the ancestraldistance punishment factor and the sibling punishment factor whencalculating the contribution of the descendant web pages. In addition,the retrieval system may add the contribution of the descendant webpages derived using the combined punishment factor to the contributionof the root web page according to the following: $\begin{matrix}{{F\left\lbrack {S\left( p_{s} \right)} \right\rbrack} = {{{\alpha(0)}{f\left( p_{s} \right)}} + {\frac{1}{{R\left( p_{s} \right)}}{\sum\limits_{u = 1}^{{h{(p_{s})}} - 1}\left\lbrack {{\alpha(u)}{\sum\limits_{p_{i_{1}} \in {R{(p_{s})}}}{\sum\limits_{p_{i_{2}} \in {R{(p_{i_{1}})}}}{\cdots{\sum\limits_{p_{i_{u}} \in {R{(p_{i_{u - 1}})}}}\frac{f\left( p_{i_{u}} \right)}{\prod\limits_{k = 1}^{u}{{R\left( p_{i_{k}} \right)}}}}}}}} \right\rbrack}}}} & (4)\end{matrix}$where F[S(p_(s))] represents the feature of the subsite with root webpage p_(s). In one embodiment, the retrieval system may use aconventional metric for generating the feature of a web page. Thefeature of a web page may include term frequency, keyword frequency,title, image size, and so on. More generally, a feature may berepresented as a feature vector describing various characteristics of aweb page. The retrieval system may generate subsite features for allsubsites of a web site.

The retrieval system may calculate the subsite feature for a subsiteusing a variety of techniques. For example, the retrieval system mayadapt the propagation-based algorithm described in U.S. patentapplication Ser. No. 11/273,715, entitled “Hierarchy-Based Propagationof Contribution of Documents” and filed on Nov. 14, 2005, which ishereby incorporated by reference. When using that propagation-basedalgorithm, the retrieval system may initially generate the feature forthe web pages without descendant web pages (i.e., leaf web pages). Theretrieval system then propagates the feature of these web pages to theirparent web pages and to their parent web pages up the hierarchy of webpages. The retrieval system may generate the feature of each ancestorweb page from the web page itself and then factor in the feature of itsdescendant web pages to provide an overall feature for the ancestor webpage. The retrieval system may generate the feature of a web pageaccording to the following equation: $\begin{matrix}\begin{matrix}{{g\left( {{F(p)},{{Child}(p)}} \right)} =} \\\left\{ \begin{matrix}{{F(p)},} & {{{Child}(p)} = \Phi} \\{{{\left( {1 + \alpha} \right){F(p)}} + {\alpha\frac{\sum\limits_{q \in {{RelChild}{(p)}}}{g\left( {{F(p)},{{Child}(p)}} \right)}}{{{RelChild}(p)}}}},} & {{{RelChild}(p)} \neq \Phi} \\{{\left( {1 + \alpha} \right){F(p)}},} & {{{Child}(p)} \neq \Phi} \\\quad & {{{and}\quad{RelChild}(p)} = \Phi}\end{matrix} \right.\end{matrix} & (5)\end{matrix}$where g(F(p),Child(p)) represents the feature of web page p, F(p)represents a function for generating the feature for web page p,Child(p) represents the child web pages of p, Φ represents the emptyset, RelChild (p) represents the child web pages of p that are relatedto the feature, | | represents the number of web pages in a set, and arepresents the weight between the feature derived from the web pageitself and the feature of the descendant web pages.

FIG. 1 is a diagram that illustrates the hierarchical relations of webpages of a sample web site. The web site 100 has a root web page 101identified by the URL “www.va.gov.” Web page 101 has child web pages111-115, with URLs “www.va.gov/disclaim.htm,” “www.va.gov/resdev,”“www.va.gov/dva.htm,” “www.va.gov/vetdata,” and“www.va.gov/spec_prog.htm.” In this example, the depth of the URLindicates the hierarchical relations of the web pages. For example, webpage 120 is a child of web page 112 and has the URL“www.va.gov/resdev/ps.” Web page 120 has child web pages 131 and 132with URLs “www.va.gov/resdev/ps/psmr” and “www.va.gov/resdev/ps/pshrd.”Web page 131 has child web pages 141 and 142 with URLs“www.va.gov/resdev/ps/psmr/mrs_programs.htm” and“www.va.gov/resdev/ps/psmr/default.htm.” In this example, web page 101is an ancestor web page common to all the other web pages, and web page141 is a descendant web page of web pages 131, 120, 112, and 101. Anyweb page of the website can be considered a root web page of a subsite.For example, web page 131 is a root web page of a subsite that includesweb pages 131, 141, and 142. One skilled in the art will appreciate thatvarious techniques may be used to identify the hierarchical relations orstructure of documents and in particular web sites. For example, thehierarchical structure of a web site may be derived from a site mapincluded as a web page of the web site or may be derived from theintra-site links between web pages.

FIG. 2 is a block diagram that illustrates components of the retrievalsystem in one embodiment. The retrieval system 230 is connected to websites 210 and user computing devices 215 via communications link 220.The retrieval system includes a search engine component 231 and a searchindex store 232. The search engine component receives queries from usercomputing devices and identifies search results based on the searchindex store. The search index store may contain a mapping of keywords toweb sites as generated by a web crawler. The retrieval system alsoincludes a re-rank result using subsite component 233, a rank ancestorsubsites component 234, a calculate subsite feature component 235, acalculate descendant contributions component 236, and a calculatedescendant contributions at level component 237. The re-rank resultusing subsite component is invoked by the search engine component tore-rank search results factoring in subsite features derived fromsubsites that have a web page of the search result as their root webpage. The re-rank result using subsite component invokes the calculatesubsite feature component for each web page of the search result. Thecalculate subsite feature component invokes the calculate descendantcontributions component to calculate the contribution of descendant webpages to the subsite feature. The calculate subsite feature componentthen combines the feature of the root web page with the contribution ofthe descendant web pages to generate the subsite feature. The calculatedescendant contributions component invokes the calculate descendantcontributions at level component for each level within the subsite tocalculate the contribution of the web pages at that level to the subsitefeature.

The computing device on which the retrieval system is implemented mayinclude a central processing unit, memory, input devices (e.g., keyboardand pointing devices), output devices (e.g., display devices), andstorage devices (e.g., disk drives). The memory and storage devices arecomputer-readable media that may contain instructions that implement theretrieval system. In addition, the data structures and messagestructures may be stored or transmitted via a data transmission medium,such as a signal on a communications link. Various communications linksmay be used, such as the Internet, a local area network, a wide areanetwork, or a point-to-point dial-up connection.

The retrieval system may be implemented in various operatingenvironments that include personal computers, server computers,multiprocessor systems, microprocessor-based systems, programmableconsumer electronics, network PCs, minicomputers, mainframe computers,distributed computing environments that include any of the above systemsor devices, and the like.

The retrieval system may be described in the general context ofcomputer-executable instructions, such as program modules, executed byone or more computers or other devices. Generally, program modulesinclude routines, programs, objects, components, data structures, and soon that perform particular tasks or implement particular abstract datatypes. Typically, the functionality of the program modules may becombined or distributed as desired in various embodiments.

FIG. 3 is a flow diagram that illustrates the processing of the re-rankresult using subsite component of the retrieval system in oneembodiment. The component loops calculating a subsite feature for eachsubsite that has a web page of the search result as its root. In block301, the component selects the next web page of the search result. Indecision block 302, if all the web pages have already been selected,then the component continues at block 305, else the component continuesat block 303. In block 303, the component invokes the calculate subsitefeature component to calculate the subsite feature for the selected webpage. In block 304, the component calculates a relevance for the subsiteof the selected web page. The component may calculate the relevanceusing a term frequency by inverse document frequency metric. Thecomponent then loops to block 301 to select the next web page of theresult. In block 305, the component reorders the web pages based on thesubsite relevance associated with each web page. The component thencompletes.

FIG. 4 is a flow diagram that illustrates the processing of thecalculate subsite feature component of the retrieval system in oneembodiment. The component combines the contribution of the passed webpage with the contribution of its descendant web pages to generate anoverall subsite feature for the subsite with the passed web page as itsroot. In block 401, the component invokes the calculate descendantcontributions component passing the root web page and receiving thecontribution in return. In block 402, the component combines thecontribution of the root web page to the contribution of the descendantweb pages according to Equation 4 and then returns.

FIG. 5 is a flow diagram that illustrates the processing of thecalculate descendant contributions component of the retrieval system inone embodiment. The component loops selecting each level of the subsiteand calculates the contribution of the web pages at that level to thesubsite feature. In block 501, the component selects the next lowerlevel of the subsite starting with the level below the root web page. Indecision block 502, if all the levels have already been selected, thenthe component continues at block 505, else the component continues atblock 503. In block 503, the component invokes the calculate descendantcontributions at level component for the selected level to calculate thecontribution of the web pages at the selected level. In block 504, thecomponent adds the contribution of the selected level adjusted by theancestral distance punishment factor to the total contribution of thedescendant web pages at the selected level. The component then loops toblock 501 to select the next lower level of the subsite. In block 505,the component normalizes the total contribution by the number of childweb pages and then returns.

FIG. 6 is a flow diagram that illustrates the processing of thecalculate descendant contributions at level component of the retrievalsystem in one embodiment. The component is passed a level and calculatesthe contributions of the descendant web pages at that level to thesubsite feature. In block 601, the component initializes thecontribution of that level. In block 602, the component calculates thedenominator of Equation _, which is the product of the number ofdescendant web pages at each level from the passed level to the root webpage. In blocks 603-605, the component loops accumulating thecontribution of each descendant web page at the passed level. In block603, the component selects the next descendant web page at the passedlevel. In decision block 604, if all the descendant web pages havealready been selected, then the component returns the contribution, elsethe component continues at block 605. In block 605, the componentaccumulates the contribution of each descendant web page divided by thedenominator and then loops to block 603 to select the next descendantweb page.

FIG. 7 is a flow diagram that illustrates the processing of the rankancestor subsites component of the retrieval system in one embodiment.The component determines the subsite feature for each web page of theresult and the subsite feature for each ancestor web page of the result.For each web page of the search result, the component selects theancestor subsite with the highest relevance and returns the root webpages of those subsites ranked according to their subsite relevance. Inblock 701, the component selects the next web page of the search result.In decision block 702, if all the web pages have already been selected,then the component continues at block 708, else the component continuesat block 703. In blocks 703-706, the component loops selecting eachancestor web page of the selected web page. In block 703, the componentselects the next ancestor web page starting with the web page itself. Indecision block 704, if all the ancestor web pages have already beenselected, then the component continues at block 707, else the componentcontinues at block 705. In block 705, the component invokes thecalculate subsite feature component to calculate the subsite feature forthe subsite with its root at the selected ancestor web page. In block706, the component calculates the relevance for the subsite of theselected ancestor web page and then loops to block 703 to select thenext ancestor web page. In block 707, the component discards all but themost relevant ancestor subsites for the selected web page and then loopsto block 701 to select the next web page of the search result. In block708, the component orders the subsites that have not been discardedbased on relevance and then returns.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described above.For example, the retrieval system may be adapted to use any of a varietyof functions to calculate a subsite feature. In addition, the ancestraldistance punishment factor and the sibling punishment factor may berepresented by a variety of linear and non-linear functions. A searchengine may rank search results according to relevance based on thesubtree features. The retrieval system may have an offline mode in whichsubtree features are calculated and stored for future use.Alternatively, the retrieval system may dynamically calculate subtreefeatures for subtrees associated with queries. The retrieval system maystore the dynamically generated subtree features so that they do notneed to be recalculated for subsequent queries. Accordingly, theinvention is not limited except as by the appended claims.

1. A system for providing a search result for a query, the search resultbeing derived from hierarchically organized documents, comprising: asearch component that identifies documents that are relevant to a querybased on a feature of the documents; a subtree feature calculationcomponent that calculates a subtree feature for an identified documentbased on a contribution of the feature of the identified document and acontribution of the feature from its descendant documents; and arelevance component that determines the relevance of the identifieddocuments to the query based on the calculated subtree feature of theidentified documents.
 2. The system of claim 1 wherein the contributionof a descendant document decreases as an ancestral distance between thedocument and the identified document increases.
 3. The system of claim 1wherein the contribution of a descendant document decreases as a numberof sibling documents to the descendant document increases.
 4. The systemof claim 1 wherein the contribution of a descendant document decreasesas an ancestral distance between the document and the identifieddocument increases and as a number of sibling documents to thedescendant document increases.
 5. The system of claim 1 including anancestor subtree feature calculation component that calculates a subtreefeature representing ancestor documents of an identified document basedon a contribution of the feature from the ancestor document and acontribution of the feature from its descendant documents and whereinthe relevance component calculates the relevance of the ancestordocuments based on the calculated subtree features of the ancestordocuments.
 6. The system of claim 1 wherein the documents are web pagesand a subtree represents a subsite of web pages.
 7. A system forcalculating subtree features for subtrees having root documents,comprising: a calculate feature component that calculates a feature foreach document within a subtree; and a calculate subtree featurecomponent that calculates a subtree feature for the subtree wherein acontribution of a descendant document of the root document decreases asan ancestral distance between the descendant document and the rootdocument increases and as a number of sibling documents of thedescendant document increases.
 8. The system of claim 7 wherein thecontribution is decreased by a factor represented by the following:α(Δl)=λ^(Δl) where a represents the decrease, Δl represents ancestraldistance between the root document and a descendant document, and λrepresents a parameter to control the amount of decrease.
 9. The systemof claim 7 wherein the contribution is decreased as represented by thefollowing:$\frac{1}{{R\left( p_{s} \right)}}{\sum\limits_{u = 1}^{{h{(p_{s})}} - 1}\left\lbrack {\sum\limits_{p_{i_{1}} \in {R{(p_{s})}}}{\sum\limits_{p_{i_{2}} \in {R{(p_{i_{1}})}}}{\cdots{\sum\limits_{p_{i_{u}} \in {R{(p_{i_{u - 1}})}}}\frac{f\left( p_{i_{u}} \right)}{\prod\limits_{k = 1}^{u}{{R\left( p_{i_{k}} \right)}}}}}}} \right\rbrack}$where h(p_(s)) represents height of a subtree with the root document ofp_(s), R(p_(s)) represents the child documents of p_(s), ∥a∥ representsthe number of elements of a, and f(p_(i) _(u) ) represents the featureof document p_(i) _(u) .
 10. The system of claim 7 wherein thecontribution is decreased as represented by the following:$\frac{1}{{R\left( p_{s} \right)}}{\sum\limits_{u = 1}^{{h{(p_{s})}} - 1}\left\lbrack {{\alpha(u)}{\sum\limits_{p_{i_{1}} \in {R{(p_{s})}}}{\sum\limits_{p_{i_{2}} \in {R{(p_{i_{1}})}}}{\cdots{\sum\limits_{p_{i_{u}} \in {R{(p_{i_{u - 1}})}}}\frac{f\left( p_{i_{u}} \right)}{\prod\limits_{k = 1}^{u}{{R\left( p_{i_{k}} \right)}}}}}}}} \right\rbrack}$where h(p_(s)) represents height of a subtree with the root document ofp_(s), R(p_(s)) represents the child documents of p_(s), ∥a∥ representsthe number of elements of a, f(p_(i) _(u) ) represents the feature ofdocument p_(i) _(u) , and where α is represented as follows:α(Δl)=λ^(Δl) where Δl represents the ancestral distance between the rootdocument and a descendant document and λ represents a parameter tocontrol the amount of decrease.
 11. The system of claim 7 wherein asubtree feature is represented by the following:${F\left\lbrack {S\left( p_{s} \right)} \right\rbrack} = {{{\alpha(0)}{f\left( p_{s} \right)}} + {\frac{1}{{R\left( p_{s} \right)}}{\sum\limits_{u = 1}^{{h{(p_{s})}} - 1}\left\lbrack {{\alpha(u)}{\sum\limits_{p_{i_{1}} \in {R{(p_{s})}}}{\sum\limits_{p_{i_{2}} \in {R{(p_{i_{1}})}}}{\cdots{\sum\limits_{p_{i_{u}} \in {R{(p_{i_{u - 1}})}}}\frac{f\left( p_{i_{u}} \right)}{\prod\limits_{k = 1}^{u}{{R\left( p_{i_{k}} \right)}}}}}}}} \right\rbrack}}}$where h(p_(s)) represents height of a subtree with the root web page ofp_(s), R(p_(s)) represents the child documents of p_(s), ∥a∥ representsthe number of elements of a, f(p_(i) _(u) ) represents the feature ofdocument p_(i) _(u) , and where α is represented as follows:α(Δl)=λ^(Δl) where Δl represents the ancestral distance between the rootdocument and a descendant document and λ represents a parameter tocontrol the amount of decrease.
 12. A computer-readable mediumcontaining instructions for controlling a computer system to identifyweb pages for a search result for a query, comprising: calculating asubsite feature for subsites of web pages based on a contribution from aroot web page of a subsite and a contribution from descendant web pagesof the root web page; and determining relevance of a subsite to thequery based on the calculated subsite feature of the subsite.
 13. Thecomputer-readable medium of claim 12 wherein the relevance of a subsiteindicates how well the subsite represents a topic distillation for thequery.
 14. The computer-readable medium of claim 12 wherein thecontribution of a descendant web page decreases as an ancestral distancebetween the root web page and the descendant web page increases.
 15. Thecomputer-readable medium of claim 12 wherein the contribution of adescendant web page decreases as a number of sibling web pages to thedescendant web page increases.
 16. The computer-readable medium of claim12 wherein the contribution of a descendant web page decreases as anancestral distance between the root web page and the descendant web pageincreases and as a number of sibling web pages to the descendant webpage increases.
 17. The computer-readable medium of claim 12 wherein thesubsite feature is represented by the following:${F\left\lbrack {S\left( p_{s} \right)} \right\rbrack} = {{{\alpha(0)}{f\left( p_{s} \right)}} + {\frac{1}{{R\left( p_{s} \right)}}{\sum\limits_{u = 1}^{{h{(p_{s})}} - 1}\left\lbrack {{\alpha(u)}{\sum\limits_{p_{i_{1}} \in {R{(p_{s})}}}{\sum\limits_{p_{i_{2}} \in {R{(p_{i_{1}})}}}{\cdots{\sum\limits_{p_{i_{u}} \in {R{(p_{i_{u - 1}})}}}\frac{f\left( p_{i_{u}} \right)}{\prod\limits_{k = 1}^{u}{{R\left( p_{i_{k}} \right)}}}}}}}} \right\rbrack}}}$where h(p_(s)) represents height of the subsite with the root web pageof p_(s), R(p_(s)) represents the child web pages of p_(s), ∥a∥represents the number of elements of a, f(p_(i) _(u) ) represents thefeature of web page p_(i) _(u) , and where α is represented as follows:α(Δl)=λ^(Δl) where Δl represents the ancestral distance between the rootweb page and a descendant web page and λ represents a parameter tocontrol the amount of decrease.
 18. The computer-readable medium ofclaim 12 wherein the subsite features are calculated independently of aquery and stored for use when identifying search results of queries. 19.The computer-readable medium of claim 12 wherein the subsite featuresare calculated based on initial search results of a query.
 20. Thecomputer-readable medium of claim 19 wherein the web pages of the searchresult are root web pages of subsites.